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Washington Centers Online Student Survey Validation Study: Surfacing 
Students’ Individual and Collective Understanding of Their Learning 
Community Experiences 


Abstract 

This article reports on findings from a mixed-methods validation study of the Online Survey of Students’ 
Experiences of Learning in Learning Communities. In the quantitative part of the study we found strong 
correlations among survey items related to faculty behaviors, student behaviors, and critical thinking. Factor 
analysis yielded four primary factors: faculty behaviors that support quality learning for all students; student 
behaviors that foster responsibility for their own learning; student behaviors that support collaborative 
learning; and, student and faculty behaviors that together create an academic learning community. For the 
qualitative part of the study, we worked with LC programs at four institutions to investigate whether students 
think about the survey items in the same way we do. These findings—and an in-depth analysis of student 
responses at one campus—led to unexpected outcomes and new insights regarding LC assessment, 
specifically what students are learning from one another. In response, we developed a companion tool to the 
online survey, the Peer-to-Peer Reflection Protocol. We conclude by addressing the potential of this guided 
discussion protocol to surface students’ experiences of individual and collaborative learning. 
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Introduction 

By winter 2014, students from 62 institutions had contributed nearly 20,000 
usable responses to Washington Center’s Online Survey of Students’ Experiences 
of Learning in Learning Communities. Four years of findings from this survey are 
a matter of public record. 1 

Highlights from survey administrations from 2009-10 to the present reveal 
the broad contours of learning community (LC) practice: most LC classrooms 
encourage student engagement and many LC students are learning the moves 
associated with integrative learning. For instance, students surveyed during the 
2012-13 academic year 2 3 report that they often or very often are encouraged to ask 
questions (89%), discuss assigned work (86%), and participate in class activities 
(86%). To a lesser extent but still noteworthy, students often or very often “work 
at connecting or integrating ideas, strategies, and skills form classes/disciplines 
included in their LC” (77%) and “reflect on how these connections lead to new 
insights or understanding” (75%). Other findings indicate that, compared to their 
integrative learning, students report fewer opportunities for collaborative learning: 
students often or very often “peer review their own or others’ work in class” 
(62%) and very often “work with classmates outside of class on course 
assignments, homework, or projects” (52%). 

Not every participating institution’s survey results replicate these national 
findings. The value of this field-based survey is that it situates students’ 
experiences of learning in LCs on any individual campus within the broader 
context of LC pedagogy and practice. 

Insofar as this online survey assessment informs professional development 
for LC programs as well as directions for the LC field, how valid is it? 

Over a six-month time frame beginning in fall 2012, we investigated this 
question, using both quantitative and qualitative methods. Findings from this 
mixed-methods approach confirm the validity of the online survey—good news 
all around. But some outcomes surprised us and led to a deeper investigation of a 
foundational assessment question: what kinds of learning do learning 
communities make possible? We realized that, as a field, we do not have an 


1 https://www.evergreen.edu/washingtoncenter/survey/surveyfindings.html 

2 See posted July 2013 report, Survey of Students’ Experiences of Learning in Learning 
Communities: Highlights from the 2012-13 Survey Administrations. 

3 Validity refers to the quality of being logically or factually sound. In the case of this validation 
research, validity is the degree to which the survey instrument assesses the concepts the research 
is attempting to measure. 
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assessment tool that surfaces the deeper collaborative learning 4 we want students 
to experience and that, according to our validation study findings, many do. 

This article begins with a brief background on the origins of the online 
survey, including what constitutes a “learning community,” given the survey’s 
purpose. We then move to an account of the validation study and the findings at 
each step in the process, from the correlations and factor analysis results to the 
qualitative analyses we conducted to find out what students think about when they 
respond to survey items. 

Our analysis of these student responses—245 in total from LC programs at 
four very different community colleges and universities—prompted us to 
reconsider both the individual and collaborative dimensions of learning in LCs. 
We discuss these findings in conjunction with insights from colleagues who 
facilitated campus conversations at the four schools: California State University- 
Sacramento, Holyoke Community College, Skagit Valley College, and Sonoma 
State University. To give readers a more nuanced description of this part of the 
validation study, we offer an in-depth analysis of the Holyoke Community 
College conversations. The summary finding and six emerging themes from 
Holyoke, along with our analysis of student responses to the validation study from 
the three other schools, led to the development of a companion tool to the online 
survey, the Peer-to-Peer Reflection Protocol (see Appendix A). 

We conclude by considering this new facilitated-discussion protocol in 
relation to collaborative learning and the assessment challenges it poses. 

Framework for the Online Student Survey 

From its inception, the Online Survey of Students’ Experiences of Learning 
in Learning Communities has been a field-based initiative. Designed to collect 
national and campus-based data about the “what” and “how” of substantive 
learning in learning communities, the survey focuses almost exclusively on 
students’ experiences of learning in LC classrooms. We resisted the pressure to 
extend this focus when early drafts of the online survey were being reviewed at 
regional meetings and the national LC conference in fall 2008. 

The genesis of the survey led to this single-minded focus. Campus teams 
who were co-researchers in Washington Center’s two-year National Project on 
Assessing Learning in Learning Communities (Lardner & Malnarich, 2008/2009) 
wanted a simple-to-administer quantitative measure to use along with the 
Collaborative Assessment Protocol for examining student work 5 that Veronica 
Boix-Mansilla had adapted for the project based on her team’s research on 


4 In Learning Better Together, Vincent Tinto (2003) describes collaborative learning as “students 
learning together.” 

5 https://www.evergreen.edu/washingtoncenter/resources/integrativelearning.html 
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interdisciplinarity (Boix-Mansilla, 2005; Boix-Mansilla & Dawes Duraising, 
2007; Boix-Mansilla & Gardner, 2003). Campuses’ use of this structured 
conversation protocol led us to rethink the relationship between interdisciplinary 
and integrative learning as well as the importance of disciplinary grounding 
(Lardner & Malnarich, 2008/2009). We also discovered the power of collectively 
examining student work in the context of curricular and assignment development. 
Even for veteran teaching teams, the quality of LC work advances in stunning 
ways (Dunlap & Suit, 2008/2009). Still, examining student work for evidence of 
integration proved to be time-consuming and too specific for program-level 
assessment purposes, unless additional assessment strategies tied to LC program 
and general education learning outcomes are in place (Pettitt & Muga, 
2008/2009). 

By the end of the national project, participants recognized that LC 
classrooms needed to become places where students could develop the integrative 
habits of mind associated with integrative learning, a characteristic identified as 
one of four essential learning outcomes for undergraduates by the Association of 
American Colleges and Universities (AAC&U) in its 2007 report, College 
Learning for the New Global Century. Our generative and non-hierarchical 
understanding of integrative learning was influenced by A Statement on 
Integrative Learning (2004), co-authored by AAC&U and the Carnegie 
Loundation for the Advancement of Teaching. This one-page account of 
integrative learning appreciates the diversity of students in higher education— 
from pre-college classes to professional programs—similar to the diversity of 
students served by the LC programs involved in the national project. As the 
statement points out, 

Integrative learning comes in many varieties: connecting skills and 
knowledge from multiple sources and experiences; applying theory to 
practice in various settings; utilizing diverse and even contradictory 
points of view; and, understanding issues and positions contextually. 
Significant knowledge within individual disciplines serves as the 
foundation, but integrative learning goes beyond academic 
boundaries. Indeed, integrative experiences often occur as learners 
address real-world problems, unscripted and sufficiently broad to 
require multiple areas of knowledge and multiple modes of inquiry, 
offering multiple solutions and benefiting from multiple perspectives. 

(n.p.) 

However, as colleagues from La Guardia Community College’s team noted, 
LC faculty were often good at “parallel play”—linking classes by theme—but not 
real integration (Burg, Klages, & Sokolski, 2008/2009). Cerritos College 
colleagues put it this way: “interdisciplinary integration needs to be grounded in 
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the disciplines, which means that it is necessary to reinforce those specific aspects 
of knowledge and thinking skills relevant to the integration” (Lardner & 
Malnarich, 2008/2009, p.13). Of the many productive shifts Boix-Mansilla 
(2008/2009) named as a consequence of this national project, teams appreciated 
that designing and assessing for intentional integration introduces a rigorous 
novice-to-developing-to-advanced trajectory into LC professional development. If 
we want our students to be able integrative thinkers, ongoing faculty 
conversations about how we scaffold opportunities for practice within the LC 
program and across departments becomes critical (Lardner & Malnarich, 2009). 

Given the challenging work of designing integrative assignments that 
address public issues without diluting the disciplinary building blocks of 
integration, colleagues wanted a quick way to find out if students were being 
invited to practice integrative learning as compared to engaging in parallel play. 
The Online Survey of Students’ Experiences of Learning in Learning 
Communities —a quantitative measure developed by Washington Center in 
collaboration with Skagit Valley College’s Director of Institutional Research— 
was designed to meet this need. Survey results, combined with findings from the 
Collaborative Assessment Protocol, would give LC programs the means to gauge 
students’ experiences of integrative learning and improve teaching teams’ 
effectiveness. In this sense, generating professional development for campuses 
and the field is in the survey’s DNA. 

The adoption of a broad definition of LCs makes the online student survey 
accessible to a spectrum of LC programs. 6 For survey respondents, three choices 
are given for “type of learning community”: either “a common cohort of students 
enrolled in two or more classes who attend these classes together (e.g. minimally 
all students in class A are in class B)” or “a cohort enrolled in a learning 
communities program in which students share a common experience, such as a 
single course, seminar, and/or integrative project,” or “other.” These fields, which 
students fill out before taking the survey, require guided instruction. Students also 
select one descriptor from a range of LC courses: pre-college (developmental) 
classes; pre-college and college-level classes; college level classes only; or 
“other.” Given the variety of curricular and co-curricular LCs across the country, 
these categories represent a consensus among practitioners of who ought to take 
the survey. 

This entry checkpoint meets the bare-bones definition of LCs in play for 
thirty years (Smith, MacGregor, Matthews & Gabelnick, 2004, p. 20). Stripping 
LCs of essential components that have made them a “high-impact practice” 
(Brownell & Swaner, 2009; Kuh, 2008) makes sense, given the purpose of the 
survey. We know that to do LCs well requires more than organizing students into 

6 The online student survey is available for use but is not in the public domain. See Guidelines for 
Participation at https://www.evergreen.edu/washingtoncenter/survey/guidelines.html. 
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cohorts. When visiting an LC classroom we ought to notice a mix of practices 
including opportunities for integrative and interdisciplinary learning, pedagogies 
of active engagement and reflection, knowledge construction through 
collaborative meaning making, and co-curricular elements, such as those 
associated with active civic engagement. But the point of a minimal definition for 
LCs is to find out if students in LC classrooms in all their various renditions— 
linked or clustered classes, coordinated studies, freshman seminar or interest 
group LCs, and living-learning communities—are experiencing the quality of 
learning that is encouraged by the pedagogical moves associated with LCs. 

In other words, the purpose of this online survey is very particular. Few 
questions veer from students’ experiences of learning. Survey items are clustered 
into four topics, and the prompts reflect this concentration: 

1. Students’ engagement in classroom activities (13 items) 

In my learning community, I... 

2. Instructors’ activities that support learning (13 items) 

Teachers in my learning community... 

3. Students’ perceptions of gains in their understanding and abilities 
(10 items) 

My participation in this learning community helps me develop 
my ability to... 

4. Students’ perceptions of cognitive activities in LC vs. other courses 
(7 items) 

In my learning community, compared to other classes, / spend more, 
less, or about the same amount of time... 

Content items in the first two sections draw on research on learning and effective 
teaching (Bransford, Brown & Cocking, 1999), insights from Washington 
Center’s assessment project, and work on integrative learning by the AAC&U and 
the Carnegie Foundation for the Advancement of Teaching (Huber, Hutchings & 
Gale, 2005). Content items in the last two sections draw on established surveys 
universities and community colleges use to assess students’ perceptions of their 
learning and engagement. Based on LC coordinators’ recommendations made 
after the first field test of the survey, we added a comparison component to the 
fourth section. 


7 
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Early vetting of the survey in classrooms in which students were doing 
integrative assignments led us to choose language accessible to students at all 
levels in their studies and to stick to the word integrate. 

Now that the survey is being used across the country by very different 
schools whose LC programs are also very different—from those well-established 
to those just beginning—we asked, how valid is the Online Survey of Students’ 
Experiences of Learning in Learning Communities ? 

Online Student Survey Validation Study 

For the online survey validation study, we used a mixed methods approach 
that resulted in both quantitative and qualitative data analyses. The quantitative 
data—both correlations and factor analysis—addressed the question: What are the 
relationships between the survey items? The qualitative data addressed the 
question: Are students thinking about the survey items in the way we expect? 

This concern with what is on students’ minds when they click very often , 
often , sometimes, or never extends early qualitative work we did with Skagit 
Valley College students before releasing the online survey for a field test in fall 
2009. In that instance, Skagit students’ “think-alouds” when reading survey items 
alerted us to the perils of using teachers’ talk and to the idiosyncratic ways 
campuses might refer to LC practice. 

These think-aloud protocol results, as well as the feedback from faculty and 
administrators from the 11 institutions who participated in the survey field test, 
helped to establish the initial content validity and face validity of the survey. 
Content validity refers to whether survey items are appropriate to the area of 
study; face validity is concerned with how a measure or instrument appears “on 
the face of it.” In other words, does the survey seem like a reasonable way to gain 
the information desired? Does it seem well-designed?. 

After three years of survey administrations, we revisited the question of 
survey validity. At the end of spring term 2012, the survey database consisted of 
9,318 responses from students in both two- and four-year institutions across the 
country and three outside of the U.S. These students were enrolled in a variety of 
LC courses and levels between the winter/spring 2010 term and winter/spring 
2012. A summary of institution and program characteristics is provided in Table 1. 
This dataset was used in the quantitative portion of the validation study described 
below. 
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Table 1 


Characteristics of Learning Communities by Institution Type 


Learning Community Characteristic 

Two-Year 

_(%)_ 

Four-Year 

__ 

Type of Learning Community 

Common cohort enrolled in 2+ classes 

82.8% 

79.7% 

Cohort sharing common experiences 

14.7% 

19.9% 

Other 

2.5% 

.4% 

Level of Learning Community Courses 

Pre-college (developmental) classes 

15.7% 

1.8% 

Pre-college and college level classes 

24.5% 

4.9% 

College level classes only 

59.0% 

92.7% 

Other 

.8% 

.6% 


Quantitative Analyses 

The multi-level approach used to examine survey validity and the 
relationships between the survey items included correlations and factor analysis. 

Correlation Analysis 

For those unfamiliar with correlations as a statistical tool, correlation 
analysis is used to examine the degree to which responses to survey items vary 
together. Correlations can be either positive or negative. If scores on one variable 
increase when the score on another variable increases, there is a positive 
relationship between the two variables. If, on the other hand, one variable 
increases while the second variable decreases, there is a negative relationship. 
Correlations range from -1 to +1. A correlation of 0.00 would mean there is no 
relationship between the two variables—an unlikely result. Generally, a 
correlation of +0.70 is considered a strong positive linear relationship, while 
+0.50 is moderate, and +0.30 indicates a weak but positive linear relationship. 
Negative linear relationships follow a similar numerical pattern. 

The correlation analysis for this study included all responses in the database 
and all 43 survey items, resulting in a very large correlation matrix that displays 
correlations between all of the survey items. All correlations among the variables 
were positive. Rather than a re-creation of the entire matrix, Table 2 presents the 
results that show moderate and strong correlations among selected survey items 
related to faculty behaviors, student behaviors, and critical thinking. While 
correlations do not denote causality, the results of the analysis suggest some 
important relationships between faculty behaviors and student behaviors. For 
example, the second section of Table 2 shows that when students work on 
integrating ideas, strategies, or skills from the classes or disciplines included in 
the LC, not surprisingly, they are also likely to reflect on how these connections 
lead to new insights or understanding. 
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Table 2 


Correlation Matrix, Selected Survey Items* 


Student Behaviors 

1 

2 

3 


1 Work with other students to examine complex issues 
during class 

2 Peer review my and other students’ work during class 

3 Work with other students on group projects during class 

.503 

.511 

.516 



Faculty Behaviors 

1 

2 

3 


1 Work on connecting or integrating ideas, strategies, or 
skills from classes (or disciplines) included in this learning 
community 

2 Reflect on how these connections lead to new insights or 
understanding 

3 Use what 1 am learning to contribute to another class 

.681 

.479 

.535 



Critical Thinking 

1 

2 

3 

4 

1 Synthesizing ideas, experiences or theories 

2 Evaluating information, methods, and arguments 

3 Integrating ideas, strategies and skills from multiple 

.550 

.502 

.564 



sources 

4 Applying theories or concepts to practical problems or 

.498 

.523 

.579 


new situations 



*AII correlations are positive and significant at the .01 level (2-tailed). 


Not included in this table, but noteworthy, is the finding that the survey item 
about the degree to which faculty “Make all students feel comfortable about 
participating in class activities” was moderately correlated with seven different 
survey items related to student engagement and learning, including: “Encourage 
students to ask questions” (.65), “Encourage students to discuss assigned work in 
class” (.57), and “Help students establish productive work groups” (.53). The 
importance of establishing a comfortable learning environment was also 
demonstrated by the moderate relationships between this item and students’ 
reports that faculty “Encourage me to explore my ideas” (.54), “Talk to me about 
my ideas” (.52), “Help me use my background knowledge and life experience to 
learn new things” (.51), and “Demonstrate how to integrate concepts and skills 
from different classes in a meaningful way” (.51). 

Factor Analysis 

For readers who may also be unfamiliar with factor analysis, factor analysis 
is a statistical technique used to identify a small number of factors from a large 
number of variables—in this case, survey items. Essentially, the factor analysis 
simplifies the correlation matrix. In this study, exploratory factor analysis was 
employed to identify the primary factors that explain the correlation matrix. The 
results from the factor analysis constitute “loading” scores for the variables 
included in the analysis. A factor loading is basically the correlation of a variable 
with a factor. 
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Only 32 of the 43 survey items were included in the validation study factor 
analysis. Due to their content-specific nature, four items in the section regarding 
students’ perceived gains in understanding and abilities (quantitative reasoning, 
speaking, writing, and critical thinking skills) were excluded from the factor 
analysis. And seven survey items related to students’ perceptions of cognitive 
activities in the LC versus other courses were strongly correlated only with other 
items in this section, so these items were also excluded from the factor analysis. 

As shown in Appendix B, four factors emerged from the analysis. The 
loading scores for the survey items and the associated factor ranged from 
moderate (0.435) to high (0.784). As a research team, we identified the four 
factors as: 

1. Faculty behaviors that support quality learning for all students, 

2. Student behaviors that foster responsibility for their own learning, 

3. Student behaviors that support collaborative learning, and 

4. Student and faculty behaviors that, together, create an academic learning 
community. 

These four factors reflect key dimensions of students’ experiences of learning in 

o 

LCs consistent with the original concepts used to create the survey. These factors 
can be used by researchers and practitioners to provide credible summary data to 
administrators, the campus community, trustees, and other stakeholders. For 
instance, a college would be able to compare benchmarks over time and/or in 
relation to other similar institutions as illustrated in Table 3. 


Table 3 

Benchmark Data for Community College X and All Two-Year Institutions, 
Fall 2010 & Fall 2011 



Community College X 

All Two-Year Institutions 


Fall 2010 

Fall 2011 

Fall 2010 

Fall 2011 

Faculty behaviors that 
support quality learning for all 
students 

3.63 

3.79 

3.49 

3.56 

Student behaviors that foster 





responsibility for their own 
learning 

3.32 

3.43 

3.26 

3.32 

Student behaviors that 
support collaborative learning 

2.85 

2.91 

2.90 

3.00 

Student and faculty behaviors 
that, together, create an 
academic learning community 

3.64 

3.88 

3.48 

3.53 


8 The factor analysis represents a statistical relationship between the survey items whereas the 
survey categories and the associated survey items represent an a priori conceptual relationship. 
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In summary, the findings from the quantitative analysis, including both the 
correlational data and the factor analysis, underscore the validity of the online 
student survey. The evidence indicates that the survey measures what it was 
designed to measure in terms of the relationship between faculty and student 
behaviors, integrative learning, and critical thinking. 

Qualitative Analysis 

For the qualitative portion of the validation study we wanted to know what 
experiences of learning came to mind when students responded to survey items. 
Given our aim to develop a simple-to-administer quantitative measure that could 
be used to complement the Collaborative Assessment Protocol for examining 
student work, we were particularly interested in items focused on integrative 
learning. 

Often the best way to learn what students think is to ask them. Fortunately, 
we were able to work with faculty and staff from LC programs at two four-year 
schools in California, Sonoma State University and California State University 
(CSU)-Sacramento, and from two community colleges, Skagit Valley College in 
Washington and Holyoke Community College in Massachusetts, to solicit direct 
feedback from students. 

While all these LC programs have stood the test of time, ranging from 20 
years to 45 years, they proved to be diverse in terms of structure, history, and the 
student population served. For instance, the two four-year LC programs are quite 
different: CSU Sacramento’s program focuses on providing transitional support 
for educationally and economically disadvantaged students from college entry to 
their senior levels; the Hutchins School of Liberal Studies at Sonoma State 
University is a virtual “school within a school,” offering a theme-based, 
interdisciplinary general education program over four years. In contrast, the two 
community college LC programs have much in common: team-taught paired 
courses, integrated coursework, and a significant portion of first-generation 
college students. Still, Skagit Valley serves a mainly rural population in the 
northwestern part of Washington and has 5,800 students, while Holyoke has 
12,000 students and is an urban institution that serves both small towns and the 
largest city in western Massachusetts. (See Appendix C for an overview of each 
of the four LC programs, the number of study participants, and selected student 
quotations.) 

At each of the four campuses, a three-step process for soliciting student 
feedback was used. Students first took the Online Survey of Students’ Experiences 
of Learning in Learning Communities. We then asked students to respond in 
writing to six items from the online survey most related to integrative learning as 
described in Table 4. 


http://washingtoncenter.evergreen.edu/lcipjournal/vol2/iss 1/1 
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Table 4 

Six Survey Items Included in Qualitative Study 

Part I. In my learning community, I: 

1. Work on connecting or integrating ideas, strategies, or skills from classes (or disciplines) 
included in this learning community 

2. Reflect on how these connections lead to new insights or understanding 

3. Use what I am learning to contribute to another class 


Part II. Teachers in my learning community: 

4. Demonstrate how to integrate concepts and skills from different classes in a meaningful 
way 

5. Assign work that asks me to connect concepts and skills from different classes to reach 
new understandings and/or applications 

Part IV. In my learning community, compared to other classes, I spend more, less, or about the 
same amount of time: 

6. Integrating ideas, strategies, and skills from multiple sources 


The directions provided to students asked: “For each of the survey questions 
below, provide one or two examples or occurrences in your learning community 
that came to mind as you were responding to the question.” Students did not know 
beforehand that they were going to do this reflection exercise. After students had 
completed this second step, faculty members (or a designee) facilitated a 
discussion with students based on their responses and recorded highlights of this 
discussion. Both the individually written responses and the discussion highlights 
were sent to the research team. 

Following an initial review of students’ written responses, it was clear that 
we needed to know more about the LC assignments, courses, and course-levels 
students were referring to as they described their learning experiences. 
Consequently, telephone interviews between the research team and the 
participating institutions were conducted to clarify program purpose and student 
characteristics. The interviews included the following questions: 

1. What was the purpose of the LC? 

2. Can you tell us a little bit about the LC? Title? Courses? Theme, common 
issue or question? Student learning outcomes? 

3. What credit level was the LC? College-level? Pre-college? Both? 

4. Is there anything about your LC students that would help us better 
understand their responses? Academic preparation? Socio-economic status? 
English language? Other? 

5. Would you share the syllabus and/or integrated assignments with us? 

These conversations provided a rich background about the programs and students 
involved in the qualitative study and helped the research team better understand 
the context for student responses. 
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Students’ Written Responses to Prompts about Integrative Learning 

The original intent of the validation study was to verify that students 
understood the survey items as the survey designers had intended. At all four 
campuses, students had specific learning situations in mind when responding to 
survey items focused on integrative learning. Their written responses map onto 
the varieties of integrative learning described in A Statement on Integrative 
Learning (2004). 

Especially interesting is the emphasis students place on the importance of 
collaborative learning as a site for integrative learning. In response to the prompt, 
In my learning community, I work on connecting or integrating ideas, strategies, 
or skills from classes (or disciplines), almost every student named “working with 
others on projects.” Examples included “developing a study guide for geology 
together,” “creat(ing) a community service project to benefit the community,” and 
“setting up study groups”—all LC assignments. A few students wrote about more 
specific connections: “ethnicity and economics,” which led to new insights about 
the “economics of racism”; “culture and rituals in anthropology,” which helped to 
make sense of “students’ cultural identities”; and, “problem-solving skills,” which 
could be applied to an “engineering class.” 

For students in this study who navigate the academy with relative ease, as 
well as those who are more advanced in their studies, the “seminar” is most often 
named as the site where robust integrative learning occurs. Students’ comments 
include the following insights: “Being able to research and read about a ton of 
different things, but then bringing it all together is something really special about 
our program. We challenge, question, and connect ideas all the time”; “I 
constantly come to new realizations about what we talked about in class/seminar”; 
“Synthesizing ideas leads to new discoveries. Connecting classes has helped me 
realize truths about myself and the world around me.” And, as one student 
indicates, integrative learning within a seminar context doesn’t require a teacher's 
oversight: “Our seminars are focused around integrating ideas and strategies and 
we do it ourselves without the teachers telling us how to pull all the information 
together as they do in other classes.” 

In response to whether 1 use what I am learning to contribute to another 
class, almost all students new to higher education and academic culture wrote 
about integration as the application of study skills and strategies learned in a 
student success class to all classes—something students were expected to do. 
They described using test-taking and reading strategies, communication and 
writing skills, time management, study group methods, and presentation skills in 
the context of various general education courses. One student reported, “I really 
didn’t focus on integrating ideas, strategies, and skills because I never thought 
about it,” underscoring the intentionality necessary to make connections from one 
class to another. For those students in a four-year, interdisciplinary, theme- 


http://washingtoncenter.evergreen.edu/lcipjournal/vol2/iss 1/1 


12 



Malnarich et al.: Washington Center’s Online Student Survey Validation Study 


oriented program, integration was both horizontal and vertical: “At the end of 
every lower division course and senior seminar we had to write a summative 
paper about our experience and tie together many things. After writing these 
‘intellectual journey’ papers I would see how the ideas worked off one another 
and were reflected in my own life. ...In most other classes I rarely synthesized 
any ideas; only learning cursory knowledge of single topics.” Other students 
referred to specific lower-division classes that they drew on for their senior 
capstone project. 

Despite differences in the “what” of learning, when asked to reflect on how 
the examples of connected or integrated learning led to new insights or 
understanding, a majority of students described “how” the experience of working 
together revealed the limits of their own experience and knowledge. As one 
student in a program designed for students new to college learning noted, “Team 
work and accepting other people’s ideas and thoughts are helpful and important. 
If I’m going to stick with only my thoughts or ideas, I may not realize my 
mistakes and/or improve or move forward. This teamwork and accepting others’ 
ideas can contribute to any classes.” Another student in the same class wrote: 
“These connections help us see the perspective of other people and help us 
discover new things: what’s believed now can lead to something new in the 
future; team work is a need, not an option.” A number of similar comments 
reflected these students’ first encounter with what A Statement on Integrative 
Learning (2004) describes as “utilizing diverse and even contradictory points of 
view.” 

More seasoned students highlighted the benefits of working together in this 
way: “You listen to people who think differently from you and use that to build 
your knowledge and understanding of the world.” Another wrote, “The different 
ideas and viewpoints of other classmates provide for a very diverse conversation 
that allows for the entire group to realize ideas they would have not otherwise.” 
And a third observed that “respecting and understanding others’ point of view 
helps you learn and think about your own beliefs and even sometimes change 
them.” One student, excited about learning from others, simply noted: “more 
involved (intellectually).” 

While students’ responses did not follow a version of the faculty shorthand 

for designing integrative assignments (“I will integrate _ with _ to 

understand/do_”), they clearly had examples of integrative learning in mind 

when they did the online student survey. 

New Insights About Collaborative Learning 

An unexpected outcome of the qualitative part of the validation study has 
been the wealth of students’ observations about their learning regardless of LC 
structure. Their accounts mirror curricular approaches associated with LCs “at 
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their best” as detailed by Smith et al. (2004, pp. 20-21). For instance, students 
referenced experiential learning and practice, intentional community building, and 
constructivist classroom experiences. In the latter case, as Smith et al. noted, 
when LCs are designed as part of an educational reform effort, students’ 
classroom experiences will be very different from those in traditional classrooms, 
primarily based on shifting assumptions about the nature of knowledge and the 
consequent implications for teaching and learning. 

Simply put, within the LC classroom the theory that knowledge is socially 
constructed is implemented through the practice of collaborative learning: the 
focus is on who is in the classroom (not just the nature of the curriculum), and on 
the curriculum as a means to encourage connected, relational, and constructed 
knowing. Smith et al. also observed that, with respect to changing views of 
knowledge and learning, “learning communities provide a significant arena for 
putting those theories into practice” (p. 22)—as evidenced, we would add, by the 
four LC programs in the validation study. 

The student writing exemplified how shifts away from traditional 
approaches to education have been implemented in their LC from the students’ 
perspective. For example, one student described the changes as a result of an 
active, contextualized learning environment: “I learned how to work with others 
instead of always working by myself all the time. I learned how to grow outside 
of me.” Another student wrote about the process of constructing knowledge 
through inquiry (rather than simply acquiring it): “One great thing I’ve 
learned...is to question everything. I’ve learned not just to accept things that are 
presented to me without questioning it. I now do this with everything in my life, 
not just in my classes and courses, but in life.” Another student described a 
changed sense of self as a result of the LC experience: “In this class, I have 
migrated from a generally uninformed green capitalist to a tentative eco- 
socialist.” 

Students also wrote about how they used the abilities developed in their LC 
in other aspects of their education: “I use my math and writing skills for other 
classes. When my teacher gave us an article about how to learn math and how you 
can be successful, I thought about how I could use those skills and practice 
outside of math.” Finally, one student illustrated both the process of inquiry and 
the importance of learning with peers: “I have learned to form opinions and 
question what I have grown up accepting. Upon entering college there is a 
moment when you don’t know whether to trust your parents’ judgment for the rest 
of your life or make your own choices. Through the seminar process I feel as if I 
have transitioned through this problem with the support of my peers.” 

Despite differences in LC programs, students describe their experiences in 
the classroom and the nature of their learning in some very similar ways. They 
reference components associated with exemplary classrooms and high-impact 
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practices: active engagement and reflection, opportunities for integrative learning, 
and collaborative learning and meaning-making. In students’ written responses to 
the six survey items and even more so in follow-up facilitated discussions, the 
multiple references made to what students learn from their peers is striking. 

As compared to more traditional educational settings, opportunities for 
students to leam collaboratively from one another are typically part of an LC 
pedagogical design. However, rarely do LC instructors explicitly assess 
collaborative learning in ways that help students see how their individual learning 
adds up to something greater when they collaborate. What do students collectively 
say about this aspect of their learning experience? And what do they identify as 
the necessary conditions for establishing a collaborative learning community, 
where knowledge is shared, developed, and applied as a collective enterprise? 

In the Holyoke Community College case study that follows, students delved 
into the collaborative aspect of their LC learning experience during the facilitated 
discussions. We turn to highlights from these conversations after briefly 
introducing Holyoke’s LC program, including how it has used the online student 
survey in its work. 

Holyoke Community College Case Study 

As one of the eleven colleges to field test the Online Survey of Students' 
Experiences of Learning in Learning Communities, Holyoke Community College 
(HCC) has continued to administer the online survey every semester to 
approximately 500 first-year and second-year students enrolled in 25 LC classes. 
These two-course learning communities feature interdisciplinary subject matter, 
collaborative learning pedagogy, team teaching, and the collaborative assessment 
of integrative learning and disciplinary grounding. Like many other institutions, 
Holyoke’s LC program is an alternative to traditional college general education 
course offerings. 

The LC program at HCC uses its survey results to assess whether LC classes 
are meeting both the program’s expected student learning outcomes, such as 
constructing knowledge using two or more disciplines, and most of the college's 
general education outcomes. Since HCC first administered the online survey in 
fall 2009, the LC program has also used survey results to develop a set of 
evolving professional development practices for teaching teams to strengthen 
student engagement in LCs and improve student success. 

Known for its work on making learning visible in LCs (Mino, 2013), HCC 
has been using documentation—the practice of observing, recording, interpreting, 
and sharing the processes and products of teaching and learning through a variety 
of media—to deepen student learning and improve LC teaching (Given, Kuh, 
LeeKeenan, Mardell, Redditt & Twombly, 2010). Both this work and the 
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validation study share a common aim: to make visible “what” students integrate 
and “how.” 

For the Holyoke administration of the validation study, 50 students from six 
different LCs at the first- and second-year levels did the written survey and then 
participated in discussions facilitated by the LC program coordinator over a one 
week period. Students’ written responses to the six survey items have been 
clustered into three findings related to integrative learning, connections between 
classes, and faculty behaviors as indicated previously in Table 4. 

In response to items related to integrative learning, the majority of students 
reported learning how to think, read, write, research, and engage in conversations 
critically, while considering multiple perspectives and using multiple sources, 
including: ecological, historical, feminist, literary, philosophical-ethical, 
scientific, social-scientific, and social justice perspectives. “An LC teaches you 
that there are different perspectives,” said one student, who added, “There are 
different ways to learn, to teach, and to think.” Other responses indicated that 
students were developing critical habits of mind while being exposed to multiple 
perspectives in LC classrooms. The majority of students also reported learning to 
“complicate matters” in their attempts to understand the “underlying issues and 
driving forces” behind events, including: capitalism as a political-economic 
system and its impact on the environment, women, immigrant groups, and the 
poor; the social, economic, political, and historical dynamics of oppression; the 
philosophy and practice of sustainability; and history as connecting the past with 
the self and the world today. Perhaps more importantly, students recognized the 
limitations of their current knowledge and perspectives, and reported experiencing 
a profound shift in their understanding of themselves and their relationship to 
others and the world around them. For example, many students expressed the 
feeling that change is possible, that their newly acquired habits of mind require 
“doing” as well as knowing, and that they can be agents of change in their own 
communities: “We worked on a social action project which involved a research 
paper on a subject/topic where we could make a difference. We also had to act on 
making change...I realized I could make a difference and plan to continue the 
‘action’ part of my project when this class is over.” 

In response to items related to connections between classes, students 
reported transferring their integrative knowledge and skills to other classes and 
their personal lives: “I would say I carry what I am learning not only to other 
classrooms, but everywhere I go. No matter the track, I feel the [LC] class makes 
you a better person.” Students also reported higher levels of motivation, effort, 
and accomplishment as they integrated their learning in LCs as compared to their 
other classes. One student wrote: 
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In my learning community, I actually spend much more time with 
homework assignments, readings, and projects assigned primarily 
because the learning approach is stimulating and more thought- 
provoking. So much so that I want to spend more time on each subject. 

My personal motivation to succeed is higher. I self impose a higher 
standard on myself. 

In response to items related to faculty behaviors , students perceived faculty 
to be integral to their developing integrative abilities, but it was the collaborative 
approach to assignments that resulted in deeply integrative learning experiences 
for almost all students. This shared view is reflected in the following response: 

The interdisciplinary approach to learning helps us to reach new 
understandings and applications because it forces students to think as a 
member of a community with some sense of belonging that drives our 
desire to work together and to help each other learn. 

Students described a variety of integrative assignments in their LCs, 
including: application exercises, reading response papers, synthesis papers, 
reflective journaling, TVEAs (topic, viewpoint, examples, and analysis exercises), 
an end-of-semester integrative research project and presentation, and service 
learning projects. Many of these integrative assignments involved out-of-class 
experiences, such as field trips to local renewable energy sites, a social action 
project to “make a difference” in their community, a service learning project with 
a not-for-profit community organization serving teen mothers, a research project 
on immigration in the city of Holyoke, or integrative science labs conducted off- 
campus. Students engaged in these experiential learning activities reported deeply 
integrative learning experiences when learning connects the LC classroom and the 
community. 

Some students did have difficulty understanding the precise differences 
between the survey questions and often parroted back the questions in the 
answers, i.e., rephrasing the question in general terms without reporting specific 
examples or illustrations of the kinds of integrative learning they engaged in. The 
facilitated discussion, however, produced rich data regarding students’ collective 
perceptions of their integrative learning experiences in LCs. For example, in 
response to the fourth item of the protocol—“Teachers in my learning 
community... Demonstrate how to integrate concepts and skills from different 
classes in a meangful way”—one student from the Carbon-Free Energy: Fantasy 
or Future LC wrote: “Talking about how each of us can make a difference in our 
lives through the things we've learned.” Compare this response to the increase in 
evidentiary details in the notes from the facilitated discussion: 
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Along with the usual academic instruction, the professors initiated 
student involvement in the classroom, field trips, and what the students 
referred to as peer teaching. The students described their LC class as 
student-centered and interactive. For example, students presented their 
final research projects (consisting of a research paper and PowerPoint 
presentation) on topics as they emerged from the text over the course 
of the semester. In this way each student became an in-class authority 
on that topic—able to use relevant texts to interact knowledgeably 
with teachers and the rest of the class. 

The capacity to deepen and extend insights, first written about on one’s own 
and then examined in the company of others during the facilitated discussions, 
wasn't completely unexpected. Many LC students are experienced in seminaring 
and thus practiced at what Mercer (2000) describes in Words and Minds as 
“thinking together” or “exploratory talk,” one form of collaborative learning: 

...partners engage critically but constructively with each other’s ideas. 
Relevant information is offered for joint consideration. Proposals may 
be challenged and counter-challenged, but if so reasons are given and 
alternatives are offered. Agreement is sought as a basis for joint 
progress. Knowledge is made publicly accountable and reasoning is 
visible in the talk. (p. 98) 

As the LC coordinator met with successive groups of students, it became 
increasingly clear that exploratory talk, or the facilitated discussion itself, 
promoted the collective construction of knowledge, thus refining, deepening, and 
transforming students’ understanding of their learning experience in LCs. When 
students engaged in the facilitated discussions, they were more articulate than in 
their written responses. The facilitated discussions encouraged further elaboration 
with examples and complicated students’ reported experiences by allowing for the 
“interrogation” of individuals’ contributions, i.e., probing, reflecting, and follow¬ 
up questioning. In these conversational give-and-takes, a similar practice emerged 
to that described by a student when talking about their classroom experience: “In 
LCs we engage, discuss, and build on each others’ ideas. LCs challenge me to go 
beyond. I need to be challenged.” This focus on creating collective knowledge 
added breadth and depth to the learning of individual students as well as the 
group. 

Not to be overlooked or underestimated, students' sense of community 
seems to be foundational to this exploratory talk as well as all other integrative 
learning activities in LCs. Mercer and Dawes (2008) elaborate: 
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In exploratory talk, then, a speaker “thinks aloud,” taking the risk that 

others can hear, and comment on, partly-formed ideas. Engaging in 

exploratory talk is therefore rather a brave thing to do, and tends not to 

happen unless there is a degree of trust within a discussion group, (p. 

67) 

As so often happens in LCs done well, what we were witnessing in the 
facilitated discussions is the interplay between the growth of collective knowledge 
and of individual knowledge, with each supporting the other, or, as Bielaczyc and 
Collins (1999) point out, we were noticing the existence of a community identity 
that became visible as the discussion progressed: “By working toward common 
goals and developing a collective awareness of the expertise available among the 
members of the community, a sense of 'who we are' develops” (p. 275). As one 
student reminded us: “A learning community is a group of people who share 
common emotions, values, or beliefs, and are actively engaged in learning 
together.” 

In summary, six themes emerged from an analysis of Holyoke students' 
responses to the validation study’s questionnaire, as evidenced by both the written 
responses and facilitated discussions: 

1. Students are learning critical habits of mind in LC classrooms. 

2. Students are changing their minds, themselves, and their communities. 

3. LC students' new knowledge, developing skills, new perspectives, and 
emerging sensibilities are being transferred to other classes and their 
personal lives. 

4. Faculty can be powerful models of integrative dispositions, habits, and 
scholarship. 

5. A collaborative approach to integrative assignments results in deeply 
integrative learning experiences. 

6. LC students engage in more integrative learning, receive more 
encouragement to do so, are motivated to do more work overall, and 
accomplish more in comparison to their other classes. 

The most significant finding to surface from the validation study at HCC 
was that collaborative learning activities and classroom conversations (either 
through student-led discussion and/or seminars) promoted the collective 
construction of knowledge, that is, students making meaning together, thus 
deepening students’ understanding of the subject matter, themselves, and the 
world around them. 

At Holyoke, facilitated discussions added a dynamic element to the 
validation study. The face-to-face exchange with peers turned out to be even more 
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generative than writing a response to a prompt on one’s own; students considered, 
then collectively built on, the insights of others. 

Follow-up phone interviews with colleagues from the other campuses 
suggested that this part of the validation study might have broader implications 
for the LC field. Our research team grew to appreciate that students’ reflective 
writing about integrative learning, in tandem with a facilitated discussion, created 
opportunities for collective reflective inquiry not only at Holyoke, but at the other 
campuses as well, especially in relation to collaborative learning. With this in 
mind, we wondered whether the collaborative dimension of LC learning could be 
more easily surfaced if we designed a protocol to organize collective reflective 
inquiry for the LC field’s use. 

A New Companion Tool to the Online Student Survey 

From the perspective of protocol design, what could we learn from students 
who participated in the validation study? Re-reading the stack of student 
responses revealed a pattern that we used to develop the Peer-to-Peer Reflection 
Protocol. 

Using an initial selection of online survey questions that focused on 
integrative learning, the written part of the validation study invited students to 
think about their experiences of learning in LCs through a particular lens—and 
they did. However, two of the six prompts generated the most thoughtful and 
prolific writing on the part of over 200 students from all four schools: 

In my learning community, I: 

1. Work on connecting or integrating ideas, strategies, or skills from 
classes (or disciplines) included in this learning community. 

2. Reflect on how these connections lead to new insights or understanding. 

Taking a cue from students’ written narratives, we based the first step of the 
two-step Peer-to-Peer Reflection Protocol on the above two prompts (see 
Appendix A). This first part of the protocol—referred to as the “post-survey 
written reflection exercise”—still asks students to provide one or two examples or 
occurrences that came to mind when they answered this part of the online survey. 

The second part of the protocol, the “facilitated discussion,” uses the format 
of a structured conversation. Its purpose is explicit: “to provide an opportunity for 
students to develop a collective understanding of their learning community 
experience by considering both the individual and collaborative nature of learning 
in LC classrooms.” This facilitated discussion has three steps. In the first, 
Introducing the Process , the facilitator reviews the facilitator role and each step in 
the reflective discussion. In the second, Discussion Agenda, facilitation for each 
point is briefly detailed: (1) listening round; (2) making connections; (3) 
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wondering; and (4) uncovering complexity. In the last step, Appreciating 
Students’ Contributions to the LC Campus Program , students are thanked for 
sharing their insights with classmates and for their contributions to improving the 
LC program on campus. 

The discussion agenda in the second part of the protocol is grounded in a 
different “reading” of students’ written responses than that done for the validation 
study. For instance, when students’ responses from all four schools are read in 
one sitting, it is evident that despite the particularities of students’ experiences 
and comments, they are clearly part of a specific learning community. Each 
response, while unique, “fits” within a larger set of responses informed by 
common learning touchstones. The student who wrote, “I feel more patient with 
life,” did so after commenting on the self-discipline learned through LC project 
work; another, referring to “the gift” of getting a job at a writing center, 
acknowledged that “when you keep practicing writing, you inevitably get better at 
it,” a reference to a writing-intensive LC. This interplay between the individual 
and community struck us as a critical entry point: everyone needs to hear the 
uniqueness of each classmate’s learning experience before noting connections or 
common themes. Hence, in the facilitated discussion, a “listening round” begins 
the conversation, followed by “making connections.” 

The next step of the discussion agenda—“wondering”—recognizes that 
some students’ accounts of learning leave us wanting to know more: “I learned to 
understand that what I believe now will, and can change in the future. I can accept 
that at times others may influence my beliefs.” Other students’ responses touch on 
aspects of their learning experience that are less visible and present opportunities 
for “uncovering complexity”—the last discussion agenda point. Lor instance, one 
student cited a specific author to illustrate how integrating ideas led to new 
insights: “David Suzuki’s argument about [the] human tendency to become so 
micro-focused on [the] smallest details, rather than [the] full picture, opened my 
mind up way beyond [the] confines of one subject and one idea.” Then, after 
describing a series of connections generated by the Suzuki insight, the student 
concluded: “Although not all the connections are immediate, when they come to 
me they stick, until a new idea or piece of information comes to me and the 
process happens again.” This student’s account opens the door to a collaborative 
examination of the moves involved in developing integrative habits of mind and 
the nature of this activity: Is the activity solitary and/or collaborative? Is it 
dependent on each student as meaning maker and/or part of a community’s 
negotiated understanding? 

If we are to arrive at “uncovering complexity,” the conversation with 
students needs to be both exploratory and intentional. In this regard, we applied a 
lesson from Washington Center’s national assessment project: use a protocol to 
structure a conversation. Along with our campus co-researchers for that project, 
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we discovered that the Collaborative Assessment Protocol, introduced by Boix- 
Mansilla, kept our collective attention focused on evidence of disciplinary 
grounding and integration in students’ writing. Without the protocol, we 
wandered off into interesting yet ultimately less productive conversations, given 
the national project’s aims. 

The structure of the Peer-to-Peer Reflection Protocol focuses on the 
individual and collaborative nature of learning in LCs. Like any newly-minted 
tool, it needs to be field-tested. We invite LC programs to use it as a companion 
tool to the online student survey or, with obvious modifications, to use it as a 
stand-alone assessment tool. Through its use—and refinement of its prompts over 
time—we hope that, as a field, we will discover more about students’ individual 
and collective experiences of collaborative learning. 

Conclusion: The Missing Piece in LC Assessment 

The mixed-methods validation study of Washington Center’s Online Survey 
of Students’ Learning in Learning Communities began as a “good housekeeping” 
endeavor: How valid are the online survey’s findings? It turns out that it meets all 
the requirements of a standard validation study and that students do have 
examples of integrative learning in mind when they respond to survey prompts. 
But, as we moved into the qualitative part of our research study, we were struck 
by an anomaly: while LCs are associated with collaborative learning, LC 
assessment of student learning, in keeping with assessment practices in general, 
elicits individual not collective responses. The online student survey is no 
exception. 

We wouldn’t have noticed this curious phenomenon—and its implications 
for LC practice—if we had simply stopped the study after collecting and 
analyzing individual students’ written responses to online survey prompts. But, 
when these responses became the basis for facilitated discussions at four very 
different LC programs, the constructivist collaborative dimension of LCs became 
more visible in the context of students’ “thinking together” and their “exploratory 
talk” (Mercer, 2000). Similar to how collaborative learning is portrayed in the LC 
literature, students referred to instances associated with peer-group learning (most 
examples) and to the social construction of knowledge as a negotiated consensus 
among knowledgeable peers (far fewer examples). 

Still, as one colleague has pointed out, there are inherent complexities in 
trying to measure and assess constructs like “collaborative learning.” What counts 
as “evidence” of collaborative learning will differ based on LC practitioners’ 
understanding of this longstanding and much emphasized feature of educational 
innovation and LC practice (see Goodsell, Maher, & Tinto, 1992). 
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In What is Collaborative Learning ? (1992), Barbara Leigh Smith and Jean 
MacGregor introduced collaborative learning as a counterpoint to the fragmented 
and routinized character of undergraduate education; collaborative learning was 
characterized as a means to revitalize teaching and learning through social and 
intellectual engagement and mutual responsibility. They defined collaborative 
learning as “an umbrella term for a variety of educational approaches involving 
joint intellectual effort by students or by students and teachers together. In most 
collaborative learning situations, students work in groups of two or more, 
mutually searching for understandings, solutions, or meanings shared by theorists, 
or creating a product” (p. 10). Smith and MacGregor’s account included a series of 
assumptions about learning from its active and constructive nature to its richly 
contextualized nature, from the value of diversity in a student body to learning’s 
affective and subjective dimensions. Their account also summarized collaborative 
learning approaches from cooperative learning to problem-centered instruction, 
peer teaching, discussion groups and seminars, and learning communities—the 
latter regarded as the most promising place to practice collaborative learning 
because a learning community “intentionally reconfigures the curriculum” (p.18). 

Today few contemporary LC classrooms follow the model of year-long 
interdisciplinary coordinated studies. And a younger generation of LC 
practitioners may be less knowledgeable about the debates of the mid-1990s that 
made LCs less an intervention strategy for student success and more a strategy for 
educational reform and institutional change. 

In the field today, collaborative learning is typically embraced as a series of 
classroom practices equated with group work and strategies for encouraging 
active engagement in the learning process rather than as a way to destabilize the 
authoritative voice of the teacher through the deliberate “social construction of 
knowledge.” In contrast, the more radical rendition of collaborative learning not 
only moves the sage off the stage but also draws on fundamentally different 
assumptions about the nature of knowledge compared to traditional higher 
education practice. Bruffee (1993), for instance, has far more in mind than 
pedagogical strategies: “Collaborative learning assumes...that knowledge is a 
consensus among members of a community of knowledgeable peers—something 
people construct by talking together and reaching agreement” (p.3). 

In his discussion, Bruffee references Thomas Kuhn’s account of knowledge 
in The Structure of Scien tific Revolutions (1962), noting that it is “intrinsically the 
common property of a group or else nothing at all.” He wanted educators to think 
about college and university education as a “process of cultural change” and 
themselves as “agents of cultural change.” He writes: “to serve effectively as 
agents of cultural change, teachers have to organize students to learn 
collaboratively.” Traces of this version of collaborative learning surfaced in the 
qualitative portion of the validation study; less radical notions surfaced in the 
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quantitative portion of the validation study (see Appendix B, “Student behaviors 
that support collaborative learning”) and were detailed through written responses 
and during facilitated discussions. 

Certainly, current items in the online student survey that reference 
collaborative learning provide useful data for LC programs and teaching teams. 
And, as a consequence of the validation study findings, we have thought about 
adding a new section: “Students in my learning community...” with a range of 
choices that reflect what we learned from students’ experiences of collaborative 
learning during the validation study. We may still do this. 

But we think that field testing the Peer-to-Peer Reflection Protocol will 
yield quite different results: for students it is an opportunity to explore the social 
and constructivist nature of learning and knowledge in one another’s company; 
for LC programs and teaching teams it is an opportunity to appreciate the deep 
and intricate connections between LCs done well and transformative learning. In 
particular, though, we hope that the field’s use of this simple to administer Peer- 
to-Peer Reflection Protocol will encourage scholarship on the collaborative 
dimension of learning in contemporary LCs as experienced and theorized by 
students and educational reformers alike. 
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Appendix A: Peer-to-Peer Reflection Protocol 

This peer-to-peer reflection protocol is a companion tool to the Survey of 
Students’ Experiences of Learning in Learning Communities. Its aim is to provide 
LC Programs with a method which invites students—in the company of peers—to 
further explore their experiences and perceptions of integrative learning as well as 
collaborative learning. 

Designed to be used after students do the online survey, the protocol builds on 
individuals’ anonymous survey responses. It has two distinct and related parts. 
The first is a written reflection exercise based on a section from the online survey 
which focuses on integrative learning. The second uses what students’ write as a 
starting-point for a facilitated, exploratory conversation where the intent is to 
deepen students’ collective understanding of their learning community 
experience. 

Peer-to-Peer Reflection Protocol (PRP) for Exploring Students’ LC Learning 
Experiences 

Part I - Post-Survey Written Reflection Exercise 

1. Once the online survey has been administered, distribute the handout in the 
box below which asks each student to provide examples or situations which 
support their rating of two questions from the survey most related to 
integrative learning. This post-survey written reflection exercise can be done 
in the same class or in a time-frame which suits the LC Program. You will 
want to add additional spacing after each prompt; the handout should be 
around one and a half to two pages in length. 


Directions: For each of the survey questions below, provide one or two 
examples or occurrences in your learning community that came to mind 
as you were responding to the question. 

In my learning community, I: 

a. Work on connecting or integrating ideas, strategies, or skills from classes 
(or disciplines) included in this learning community 

b. Reflect on how these connections lead to new insights or understanding 
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2. After all students have had a chance to write responses, facilitate a discussion 
with students based on their responses and insights (see page 2). This can be 
done in the same class or in a time-frame which suits the LC program. We 
recommend that facilitation be done by a member of the teaching team or 
someone knowledgeable about the LC class. 

3. You may want to note down or record highlights of the discussion for future 
in-class use and/or professional development conversations. 

Part II - Facilitated Discussion 

The purpose of this part of the protocol is to provide an opportunity for students 
to develop a collective understanding of their learning community experience by 
considering both the individual and collaborative nature of learning in LC 
classrooms. 

1. Introducing the process 

The facilitator reviews the role of the facilitator, i.e. to make sure the conversation 
stays focused on each step in the reflective discussion: 1) listening round; 2) 
making connections; 3) wondering; and 4) uncovering complexity. 

2. Discussion agenda 

Listening round'. One by one, without interruption, students share examples or 
situations from their written responses in a round robin fashion. 

Making connections : Once everyone has shared their responses, the facilitator 
asks students to point out any connections or common themes given the range of 
experiences and situations. 

Wondering'. Students are invited to raise questions or name anything they are 
curious about in relation to the kind of learning their fellow students are 
describing. 

Uncovering complexity. The facilitator invites everyone present to share any 
thoughts they have about “what lies beneath the surface” 1 of their experience. 

3. Appreciating students’ contributions to the LC Campus Program 

The facilitator thanks students for sharing their insights with classmates and their 
contributions to improving the LC program on campus. 

1 This expression is from Cultures of Thinking, Harvard Project Zero: 
http://www.ronritchhart.com/COT_Resources.html. 
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Appendix B: Summary of Items and Factor Loadings 1 


Loading 

Factor Name and Items 

Faculty behaviors that support quality learning for all students 

.729 

Encourage me to explore my ideas 

.719 

Encourage students to ask questions in class 

.714 

Encourage students to discuss assigned work in class 

.705 

Make all students feel comfortable about participating in class 
activities 

.704 

Talk to me about my ideas 

.674 

Help me use my background knowledge and life experiences to learn 
new things 

.672 

Demonstrate how to integrate concepts and skills from different 
classes in a meaningful way 

.667 

Help students establish productive working groups 

.659 

Show me how to evaluate the strengths and weaknesses in my work 
as a basis for improvement 

.635 

Make the goals and vocabulary of learning communities clear 

.606 

Assign work that asks me to connect concepts and skills from 
different classes to reach new understanding and/or applications 

.578 

Encourage me to seek out other resources on campus (library, math 
center, writing center, learning center, student services, financial aid, 
etc.) 

.539 

Encourage me to plan the next steps in my education with a 
counselor or advisor 

Student behaviors that foster responsibility for their own learning 

.755 

Take responsibility for my own learning 

.734 

Be successful in future courses and programs 

.730 

Persist when faced with academically challenging work 

.682 

Identify the learning strategies that are most effective for me 

.624 

Connect my learning in school to problems and issues in my local 
community and the world 

.525 

Work effectively with others to complete assignments/projects 

Student behaviors that support collaborative learning 

.784 

Work with other students on group projects during class 

.708 

Peer review my and other students' work during class 

.634 

Work with other students to examine complex issues during class 

.609 

Present my work, or work done as part of a group, to the class 

.533 

Work on reading, writing and/or problem-solving assignments during 
class 
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Student and faculty behaviors that create an academic learning community 

.726 

Participate in class discussions or seminars 

.704 

Ask questions in class 

.582 

Reflect on how these connections lead to new insights or 
understandings 

.545 

Work on connecting or integrating ideas, strategies, or skills from 
classes (or disciplines) included in this learning community 

.488 

Discuss ideas from this learning community with family members, 
co-workers, or other students, etc. 

.435 

Use what I am learning to contribute to another class 


'Method: Thirty-two (32) survey items selected. Extraction Method: Principal Component 
Analysis. Rotation Method: Varimax with Kaiser Normalization. Rotation converged in seven 
iterations. 
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Appendix C: Overview of Qualitative Validation Study Institutions 
California State University, Sacramento (CA) 

The California State University (CSU) Educational Opportunity Program (EOP), 
established in 1969, is designed to increase the access, academic success, and 
retention of educationally and economically disadvantaged students. CSU 
Sacramento’s EOP offers transitional student support through a 5-week summer 
bridge, a fall LC, a spring LC focused on career success, and an ARISE Project 
for sophomore through senior level scholars. Other programs provide additional 
academic and social support such as the College Assistance Migrant Program 
(CAMP) for students from migrant and seasonal farm worker backgrounds. 

Study Participants: 142 students from three different LCs. 

Notable Student Quotes: 

. These connections lead me to feel more comfortable when talking in front of 
the class. It also helps me understand different aspects of really common 
things. I kind of saw these things from different perspectives. 

. I understand what college is all about, and how the system works. I became 
more and more independent on how to study for test. 

. I learned how to work with each other instead of always working by myself all 
the time. I learned how to grow outside of me. By doing/using team work, and 
learning how to communicate with others because I’m usually a quiet and shy 
person who just listen and don’t talk at all. 

Holyoke Community College (MA) 

For two decades Holyoke Community College (HCC) has offered two-course 
interdisciplinary LCs featuring interdisciplinary subject matter, collaborative 
learning pedagogy, team-teaching, and integrated assessment. HCC offers 25 LCs 
each year to approximately 500 first-year and second-year students as an 
alternative to traditional college general education course offerings. 

Study Participants: 50 students from six LCs at the first and second year levels. 

Notable Student Quotes: 

. A learning community is a group of people who share common emotions, 
values, or beliefs, and are actively engaged in learning together. 

. I would say I carry what I am learning not only to other classrooms, but 
everywhere I go. No matter the track, 1 feel the [LC] class makes you a better 
person. 

. In this class, I have migrated from a generally uninformed green capitalist to a 
tentative eco- socialist. 


31 



Learning Communities Research and Practice, Vol. 2 [2014], Iss. 1, Art. 1 


Skagit Valley College (WA) 

Skagit has been offering curricular learning communities since 1986. Students’ 
participation in LCs has been a requirement for the transfer degree since 1993. 
Most college-level LCs are paired, fully-integrated, team-taught courses with both 
faculty in the classroom at all times. Classes in developmental LCs are typically 
offered back-to-back, with overlapping integrative assignments. 

Study Participants: 7 students from a developmental math and reading LC pair. 

Notable Student Quotes: 

• 1 use my math and writing skills for other classes. When my teacher gave us an 
article about how to learn math and how you can be successful, and I thought 
about how I could use those skills and practice outside of math. 

• Story problems seemed to be everyone’s nightmare so it was great doing a 
group project and also having to write out our steps. 

• Group project when teachers were both gone. 

Sonoma State University (CA) 

Established in 1969, the interdisciplinary, theme-oriented Hutchins School of 
Liberal Studies offers lower division students an alternative general education 
program, and upper division students a major in Liberal Studies leading to a 
Bachelor of Arts (BA) degree. Hutchins also offers a minor in Integrative Studies, 
a 4-year blended program leading to a BA, and a Multiple Subject Teaching 
Credential. Seminars are a key feature of all Hutchins programs. 

Study Participants: 46 students at various levels in their Hutchins studies. 

Notable Student Quotes: 

• All of our final projects, papers, etc. contain some element of 
extending/expanding on a topic. Often times you can synthesize other course 
material with Hutchins assignments, owing to the freedom within the 
assignment. If you cannot synthesize your education, perhaps you missed out 
on your higher education. 

• I have learned to form opinions and question what I have grown up accepting. 
Upon entering college there is a moment when you don’t know whether to trust 
your parents’ judgment for the rest of your life or make your own choices. 
Through the seminar process I feel as if I have transitioned through this 
problem with the support of my peers. 

• One great thing I’ve learned in Hutchins is to question everything. I’ve 
learned not just to accept things that are presented to me without questioning 
it. I now do this with everything in my life, not just in my classes and courses, 
but in life. 
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